Such a shaving apparatus is known, for example from Netherlands Patent Application No. 8600154 which corresponds substantially to U.S. Pat. No. 4,711,028 issued Dec. 8, 1987. The disadvantage of this prior-art construction is that when the shaving head is removed from the apparatus, for example for the purpose of cleaning, the shaving unit and the retaining plate are also removed from the remainder of the apparatus. For example, for cleaning the shaving head further disassembly is required, in which case it is not unlikely that the parts are damaged or mislead. Moreover, if the apparatus comprises a plurality of shaving units, a shaving member of a shaving unit may be interchanged with that of an adjacent unit, which is undesirable.